The present invention relates to apparatus for receiving and decoding coded information such as coded pictures or sound, more particularly to improved methods of handling coded information corrupted by transmission errors.
Moving pictures accompanied by sound are transmitted over telephone networks, computer networks, and other communication channels in videophone systems, videoconferencing systems, and video-on-demand systems. These communication systems are usually organized into hierarchical layers, including a coding layer, a transmission layer, and a physical layer.
In the coding layer, digitized pictures and sound are compressively coded at the transmitting terminal, and decoded at the receiving terminal. International standards applying to the coding layer include, among others, recommendations H.261 and H.263 of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T), and the MPEG-1, MPEG-2, and MPEG-4 standards of the Moving Picture Experts Group. The MPEG standards have been adopted by the International Organization for Standardization (ISO).
In the transmission layer, coded data received from the coding layer are prepared for transmission on the physical layer, often by being divided into transmission units known as packets, and data received from the physical layer are passed back to the coding layer to be decoded, with appropriate processing such as the removal of packet headers. There are also international standards and de-facto industry standards applying to the transmission layer. Internet standards such as the Real-Time Transport Protocol/User Datagram Protocol/Internet Protocol (RTP/UDP/IP) and Transmission Control Protocol/Internet Protocol (TCP/IP) are well known, and there is a relevant ITU-T recommendation (H.223) directed toward multiplexing for low-speed terminals. Incidentally, the transmission layer is referred to as the adaptation layer in recommendation H.223.
The physical layer includes the transmitters, receivers, cables, wireless links, switches and other facilities that actually transmit the data. These facilities are prone to various types of transmission errors. For example, bit values may be altered by noise, particularly on wireless links; bits may be lost because of synchronization problems; and entire packets may be lost because of congestion in switching facilities. The terms xe2x80x98errorxe2x80x99 and xe2x80x98transmission errorxe2x80x99 will be used below to refer to all of these types of errors, including packet loss.
Even minor errors can have serious effects on the decoding of coded picture data. For example, the moving-picture coding standards mentioned above employ variable-length codes. A single bit error can disguise the locations of the boundaries between codewords, causing the decoder to lose synchronization with the variable-length code and decode all data from the point of the error onward incorrectly.
Various measures are taken in the coding layer to limit the effect of errors. One measure is to insert synchronization words, having a unique bit sequence that could not be produced by any combination of variable-length codewords, at regular intervals in the coded data. If the decoder loses synchronization because of an error, it can resynchronize by recognizing the unique bit sequence of a synchronization codeword. In moving-picture transmission systems, a synchronization codeword or start code is generally placed at the beginning of each frame. In some systems, each frame is divided into smaller coding units, and a synchronization word is inserted between each two of these coding units, limiting the immediate effect of an error to one coding unit. The coding units are variously known as slices, groups of blocks, and video packets; the term xe2x80x98slicexe2x80x99 will be used generically below to refer to any coding unit smaller than a frame.
In the coding layer, errors can be detected by the occurrence of illegal values and the violation of coding syntax rules. When an error is recognized, steps are taken to conceal it. In the simplest error concealment method employed with moving-picture data, the entire frame or slice in which the error occurred is replaced with data from the preceding frame, or the corresponding slice in the preceding frame. Other error-concealment methods discard only part of a slice, such as the part following the error, or replace the slice with data taken from other parts of the preceding frame, guided by motion vectors taken from the preceding frame.
Error countermeasures are also taken in the transmission layer. In some systems, a redundant error-correcting code is added to the transmitted data, enabling the receiving terminal to detect and correct errors if the errors are not too severe. If the bit error rate in the physical layer is low enough, a modest amount of redundancy enables most errors to be corrected and substantially all errors to be at least detected in the transmission layer.
Other standard practices are to add sequence numbers to packets, so that packet loss can be detected, and to add check bits such as cyclic redundancy check bits (CRC bits) to each packet, so that packets corrupted by transmission errors can at least be identified as erroneous. If a packet is lost or corrupted, the receiving device can then request retransmission of the packet, waiting until each packet is correctly received before passing the data on to be decoded in the coding layer. Sequence numbers also enable packets to be passed to the coding layer in the correct order, even if they arrive out of order, because of taking different routes through a network, for example.
Moving pictures and sound are often transmitted in real time, however, so the transmission layer may not be able to wait for late packets to arrive, or for erroneous packets to be retransmitted. The overall result is that even when the transmission layer can detect a transmission error, it cannot always correct the error.
When a layered system of the type described above is designed, the conventional practice is to design the separate layers independently, so that, for example, the transmission layer is compatible with any type of physical layer, and with any type of coding layer, and the coding layer is compatible with any type of transmission layer. A consequence of this independence is that the coding layer cannot rely on the transmission layer to detect errors or provide information about missing or erroneous data. Accordingly, even when such information is available in the transmission layer, it is not used in the coding layer.
This is obviously inefficient, and the inefficiency has various undesirable consequences, including reduced picture quality, reduced sound quality, added hardware and software costs, increased processing time, increased power consumption, and a degree of error immunity that, in many moving-picture transmission systems, cannot be considered robust. It is particularly unfortunate that the coding layer is sometimes unable to detect errors that have already been detected, although not corrected, in the transmission layer. From the user""s point of view, what matters is not the independence of the layers but the quality of the reproduced picture or sound, and in conventional systems there is considerable room for improvement.
It is accordingly an object of the present invention to improve the error-handling functions of a layered system that receives and decodes coded data.
Another object of the invention is to speed up the processing the received data.
Another object is to reduce power consumption by the apparatus that receives and decodes the data.
A more particular object is to prevent serious degradation of picture quality caused by errors in the transmission of coded moving-picture data.
The invented method of receiving and decoding coded data concerns a system having a transmission layer and a coding layer, in which the coded data are divided into transmission units which are received separately in the transmission layer. The transmission layer detects errors in each transmission unit and converts the received data to coding units, which have a definite relation to the transmission units. The coding units are decoded separately in the coding layer.
According to the invented method, the results of error detection performed in the transmission layer are used when the coding units are decoded in the coding layer.
Preferably, the transmission layer provides the coding layer with error information explicitly indicating which of the coding units are free of errors.
This error information may be used by, for example, skipping error checks of error-free coding units in the coding layer; performing error concealment when an error detected in the transmission layer is not detected in the coding layer; or avoiding the use of possibly erroneous data as reference data in predictive decoding. The error information can also be used to position coding units correctly, or to distinguish between correct and incorrect values when the same information appears repeatedly in different coding units. If the coded data constitute an encoded moving picture, the error information can be used to place each coding unit in the correct frame of the moving picture. If a coding unit is retransmitted, the error information can be used to recognize cases in which all received versions of the coding unit are erroneous, so that steps can be taken to select the least erroneous received data.
The invention also provides apparatus for receiving and decoding coded data, employing the invented method.